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ABSTRACT 

The basic data requirements for making educational 
program decisions in an accountability- or iented setting with the 
school principal as the focal point of the process are detailed in 
this report. The emphasis is on collecting and disseminating data now 
available within the school system. These data fall into six 
categories; school descriptors, student data, data about teachers, 
data about other staff members, administrator data, and classroom 
descriptors. Such data, together with a description of the 
instructional programs, provide the basis for evaluating individual 
programs and considering alternative programs to achieve a stated 
objective. With this information readily available, the basis for 
educational decisionmaking is improved. Also, the administrator who 
is accountable to the community, can make explicit the rationale for 
decisions affecting the programs for which he is responsible. (Other 
reports in this series are available as: LI 003908 and LI003910 
through 003912). (Author) 
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District under Contract 2410. Views or conclusions contained 
in this study should not be interpreted as representing the 
official opinion of LAUSD. 



2 



Copyright © 1971 
Published by The Rand Corporation 



2 

VOLUME II 
DECEMBER 1971 
R'831/UCS 




rremitd lor he lb lade 
Mfled scfeni narict 



Rand 

SANTA MONICA CA 90406 



-iii- 



PREFACE 




In response to community, legislative, and student pressures, school 
administrators have recently begun to examine the potential of modern 
management tools and practices. This search for techniques that might 
function effectively in an educational context led to the adaptation of 
such methods as program budgeting and accountability. Another tool fre- 
quently chosen for educational assistance is the modern management in- 
formation system, a computer-based aid to planning and decisionmaking. 

In late 1970, the Los Angeles Unified School District (LAUSD) con- 
tracted with The Rand Corporation to design such an information system 
in support of educational management. The system is specifically in- 
tended to aid planning and decisionmaking (through implementation of 
accountability and program budgeting) in schools partially supported by 
Title 1 provisions of the Elementary and Secondary Education Act. 

This report indicates the shape of the accountability system likely 
to be used in the district and describes the data appropriate for eval- 
uating both program and classroom performance. It is the second in a 
series of reports describing the research carried out. The other vol- 
umes in the series are the following: 

o J. A. Farquhar and B. W. Boehm, An Information System for Ed- 
ucational Management , Vol. I: Design Considerations , R-930-LACS. 

Defines near-term information system requirements, design guide- 
lines, major design constraints, and information needs of ed- 
ucational decisionmakers. 

o T. S. Donaldson, An Information System for Educational Manage- 
ment , Vol. Ill: Data Requirements for Evaluation: A Review 

of Educational Research , R-932-LACS. Reviews and discusses 
the literature concerning student evaluation, providing direc- 
tion for eventual information system growth. 

o J. A. Farquhar, D. H. Stewart, J. Lombaerde, An Information 
System for Educational Management , Vol. IV: Functional De- 

sign* R-933-LACS. A functional description of the proposed 
information system, specifying input and output data, file 
formats, and necessary processing. 
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o J. A. Farquhar, I. M. Iwashita, S. H. Landa, An Information 
System for Educational Management, Vol. V: A design for Im- 

plementation, R-9 34-LACS. Describes and discusses alternative 
hardware, software, and support configurations that might pro- 
vide the desired services, and the costs and benefits of each, 
o L. A. Dougharty and S. A. Haggart, An Information System for 
Educational Management, Vol. VI: An In-Service Training Pro- 

o gram, R-935-LACS. Describes the education and training require- 
ments for educational administrators charged with effective 
use of program budgeting, accountability, and the designed in- 
formation system. 
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SUMMARY 

This report details the basic data requirements for making educa- 
tional program decisions in an accountability-oriented setting with 
the school principal as the focal point of the process. The emphasis 
is on collecting and disseminating data now available within the school 
system. These data fall into six categories: school descriptors, stu- 

dent data, data about teachers, data about other staff members, admin- 
istrator data, and classroom descriptors. Such data, together with a 
description of the instructional programs, provide the basis for eval- 
uating individual programs and considering alternative programs to 
achieve a stated objective. 

With this information readily available, the basis for educational 
decisionmaking is improved. Also, the administrator who is accountable 
to the community, can make explicit the rationale for decisions affect- 
ing the programs for which he is responsible. 
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I. INTRODUCTION 

— % 

This report attempts to specify the data required for decisionmak- 
ing about educational programs. The administrator faced with decisions 
about which programs should be adopted, which expanded, and which cur- 
tailed, needs comparable data about each program under consideration. 

At a minimum, a detailed description of each program is required, as 
well as a measure of program effectiveness, a description of the target 
population, and a specification of the program's resource requirements. 

The Los Angeles Unified School District (LAUSD) is moving toward 
both further decentralization and implementation of an educational ac- 
countability system. These simultaneous and related trends, with their 
consequent information needs, have shaped the design of the evaluation- 
data specification. In view of these trends, the school principal be- 
comes the logical choice to make decisions about the programs most suit- 
able to the needs of his community; his information needs, therefore, 
must be given primary consideration. He needs evaluative data about 
all programs in his school so that he will have a sound basis for making 
program decisions. He also needs a framework within which to collect 
and disseminate whatever data may be required by other decisionmakers 
in the LAUSD. 

Section II defines the role of evaluative data and discusses the 
accountability flow. Section III presents the data requirements to 
support program evaluation, and Sec. IV illustrates the use of evalua- 
tion data in meeting Title I requirements. 
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II. THE FRAMEWORK FOR EVALUATIVE DATA 

In its most general sense, accountability is a concept that gives 
focus and direction to decisionmaking and requires a systematic delin- 
eation of responsibilities within all organizations. The key to account- 
ability implementation is an information system that provides rapid, 
timely feedback to the decisionmaker about the progress being made to- 
ward attainment of system objectives. The heart of the information sys- 
tem is the collection, analysis, and dissemination of evaluation data. 

Evaluation is a monitoring process that measures outputs of the ed- 
ucational system and systematically relates them to -.nputs so that the 
educational administrator can make the necessary decisions about program 
adoption, expansion, or curtailment. 

Decisionmaking capability is improved by the availability of rele- 
vant data about the programs under consideration. At the same time, 
such data provide firm support when decisions are made public. Educa- 
tional decisionmakers are often suspected of making capricious decisions; 
the ability to make available for public scrutiny the data on which 
they base their decisions can help to dispel this image . 

THE ACCOUNTABILITY FLOW 

The superintendent of a school district is responsible to the Board 
of Education for the operation of the schools. This implies responsi- 
bility for both the allocation of available resources and the educational 
attainment (a broader concept than achievement) of the students in his 
system. In a large school system, however, the superintendent cannot 
be aware of the operating details of each classroom in each school. Ac- 
countability, therefore, requires the superintendent to delegate au- 
thority and responsibility through the supporting administration to the 
individual classroom and to provide for feedback from the classroom 
to the superintendent along the same path. An explicit flow of respon- 
sibility, and hence of decisionmaking, is one of several keys to achiev- 
ing equitable accountability. Each administrator is, ultimately, ac- 
countable to the superintendent for accomplishing those responsibilities 
delegated to him. 
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For operating purposes, it seems necessary to choose foci for ac- 
countability that are large enough to permit sufficient freedom of ac- 
tion for operating purposes and small enough to permit adequate parti- 
cipation in the decisionmaking process. The logical unit around which 
to center accountability is the school, with the principal as the ac- 
countable administrator. Making the principal the focus of account- 
ability is in no way intended to shift the ultimate burden of responsi- 
bility from the superintendent. Rather, it is intended as an operational 
(or managerial) device to improve the system’s efficiency. 

THE SCHOOL AS THE UNIT FOR ACCOUNTABILITY 

The principal, obviously, should be involved in setting the goals 
for his school '-.i his participation in the decisionmaking process, a 
basic principle ; »f accountability, is to be achieved. It is unrealis- 
tic to expect all schools of a given type to have the same goals since 
they serve communities with different expectations from, and demands 
on, the educational system. What is generally accepted as a norm for 
educational attainment in a white, middle-class neighborhood may be ut- 
terly inappropriate, both in terms of level and content, for a school 
with a large enrollment of students, with other backgrounds. Goals and 
objectives must reflect the expectations of the students, parents, those 
responsible for providing educational services, and community leaders. 

As a professional educator, the principal knows what attainment is rea- 
sonable under existing operational constraints. He probably also under- 
stands the improvement that might be expected with a different mix of 
resources. In addition, he is attuned to the desires of his community 
and should be able to translate these into a set of reasonable educa- 
tional objectives. 

FOCUS OF ACCOUNTABILITY 

Once the principal has been established as the focus of the account- 
ability system, the following two questions need to be addressed: 

o To whom is he accountable? 

o For what is he accountable? 
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He is accountable to the community for his school's progress toward its 
objectives. He is accountable to the Zone Superintendent. If his is 
a Title I school, he is accountable to the Director of Specially Funded 
Programs. He is also accountable to the school staff for procuring the 
resources with which to implement their program, and just as the princi- 
pal is accountable to his staff, so the Zone Superintendent is account- 
able to the principal for supplying needed resources. 

Implications of the System 

Judicious accountability requires that persons responsible for 
achieving objectives participate in the decisionmaking process. This 
means that the principal and his staff must agree on objectives and 
the means of attaining them, within the following constraints: 

o A fixed operating budget. 

o District personnel policies. 

o The state education code (unless some section is waived for a 
special program) . 

o Data requirements of the district and/or state or Federal 
agencies . 

On the other hand, the principal and his staff must have the lati- 
tude necessary to carry out the program they design. Basically, this 
means freedom — within the budget allocated to the school — to specify 
the resource mix (personnel, equipment, materials) necessary for program 
implementation. This puts a requirement on the Zone Superintendent (his 
accountability to the principal) to provide the resource mix so that 
the principal can pursue his objectives. 

PROGRAM EVALUATION 

Basically, the data specified in this report all address one issue: 
improving the educational administrator's ability to make decisions 
about his programs. If he is to be accountable for his decisions, he 
must be able to assemble quickly the data on which they were based. 
Evaluation is the management tool that enables the decisionmaker to mea- 
sure progress toward stated objectives. By relating outputs of the 

■ \ -9 ' 
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educational process to inputs, he can determine what degree of success 
is achievable by a given resource mix for a defined target population. 
Such data allow him to make the necessary decisions about the programs 
for which he is responsible. 

Three kinds of program decisions need to be made: adoption, ex- 

pansion, or curtailment. Program adoption deals with the basic ques- 
tion of continuing an ongoing program or changing it. This type of 
decision often arises because a decisionmaker hears about another pro- 
gram, either in his own district or outside it, that is more successful 
than one he is currently conducting. In order to make a decision, he 
needs to know (1) the characteristics of the target population, includ- 
ing their past performance and the performance achieved in the program 
being considered; (2) the dollar cost of the program; and (3) the re- 
source requirements, in terms of kind and number of personnel, physical 
facilities, and equipment and materials required. He can then compare 
these factors with the cost, resource mix, and effectiveness of his own 
program and decide whether to continue his program or adopt the new one. 

There are times when the decision involves expansion of a success- 
ful program, or curtailment of a less successful one. The desire to 
curtail a program often leads to the decision to adopt a new one. In 
this , case, the decisionmaker is likely to search for a new program. 

In addition, when considering alternatives, continuing a program should 
not be overlooked as a decisionmaking option. 
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III » EVALUATION DATA REQUIREMENTS 

Even though the evaluation design is, ultimately, to support account- 
ability and/or program budgeting, it is logical to start by specifying 
the information that will serve the present systems. Collateral dis- 
cussion will make it apparent that the very data that are appropriate 
for use in the present school system will continue to be useful elements 
of an information system designed with some regard for the demands of 
accountability and decentralization. In this context, evaluation serves 
two separate but overlapping purposes: a management function and a re- 

search function. This report is concerned with the management function.,* 

Furthermore, much of this design is intended to provide a descrip- 
tion of the educational environment as it exists today so that the deci- 
sionmaker has available baseline data against which to measure the im- 
pact of changes. Many of the same data serve the whole management 
hierarchy; the data become increasingly aggregated as they move through 
the hierarchy from the classroom to the superintendent. 

The data considered here are those that were available within the 
school district at reasonable cost. These data may be of limited util- 
ity because of the difficulties inherent in managing educational pro- 
grams to account for individual differences; however, accurate data 
correctly used are more useful than no data at all. Until educational 
research can help identify the interrelationship among teachers and 
their methods and students and their learning styles, and until these 
data can be used to improve program planning, the available data must 
be used to best advantage. In the long run, equitable operation of an 
accountability system depends oft the reliability 'and appropriateness 
of the data. 

As a management tool, evaluat-ion-sferves accountability by assess- 
ment and identification. That is, it provides data about the current 
status of the system and identifies for further investigation those 



T. S. Donaldson, An Information System for Educational Management, 
V°l. HI* Data Requirements for Evaluation: A Review of Educational 
Research, The Rand Corporation, R-932-LACS, December 1971, concerns the 
research function. 
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aspects of the system that are notable either for lack of success or 
for unusual success. Assessment of the current status is essentially 
a bookkeeping function — a report on where the school system is with 
respect to student achievement at some point in time, usually the end 
of the school year. It is generally made by subject, by grade, and by 
school, as illustrated in Table 1, which presents reading data for Title 
I schools. It is possible, by dint of much study, to extract a great 
deal of information from this kind of report. One can compare the re- 
sults over time, by grade, and by school. One can also trace a group 
of students in the same school as they progress through the grades if 
the data are grouped for this purpose, as in Table 2. 

Identification of unsuccessful areas could be greatly simplified 
by generating an exception report that would present only the data re- 
quiring the attention of the decisionmaker. He could specify the cri- 
teria for inclusion in the report by defining what scores would be low 
enough to indicate that a given school's achievement in some grade war- 
ranted attention in light of comparable schools' performance. Or he 
might want reports only for those schools where achievement in a grade 
was appreciably lower than it had been the previous year. Again, his 
choice of criteria dictates the boundaries for inclusion. 

The same kind of data could be available to the principal of a 
school. In an elementary school, he should be able to see year-end 
achievement data by subject, by grade, and by class. The data from any 
one elementary school are not so cumbersome as to suggest an exception 
report. At the secondary level, a principal should have year-end data 
by subject, by grade, and by classroom. He will need to bear in mind 
that each class represents a more-or-less different mix of students. 

But he should have an over-view of his school's achievement. 

Identification of successful areas entails the same procedures as 
identifying troublesome areas; where data are numerous, exception re- 
ports should be made. 

The principal of a school and the administrator of a group of schools 
or programs, have the same kinds of needs: to be alerted to areas where 

i 

progress toward a goal is less than anticipated and to be aware of areas 
where it is greater than anticipated. In either case, they will vrant to 
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READING SCORES FOR TITLE I SCHOOLS 
Stanford Reading Test .National Percentiles 
(By Grade Levels) 





School 




Grade 


One 






Grade 


Two 




Grade Three 




Code 


1966 


1967 


1968 


1969 


1966 


1967 


1968 


1969 


1967 


1968 


1969 


}. 


103 


2 


3 


5 


5 


5 


3 


5 


8 


9 


7 


15 




106 


2 


2 


3 


10 


2 


7 


5 


9 


5 


4 


12 




107 


5 


7 


6 


11 


5 


5 


6 


6 


18 


15 


11 


r 


116 


2 


2 


5 


4 


4 


4 


5 


5 


1 


5 


8 




120 


3 


4 


3 


7 


3 


4 


5 


8 


3 


14 


11 


>■ 


126 


7 


5 


6 


4 


7 


7 


12 


8 


6 


15 


9 


t' 


127 


3 


5 


9 


6 


5 


5 


5 


5 


7 


6 


10 


f 

l. 


128 


2 


1 


6 


8 


2 


2 


3 


5 


4 


9 


4 


?. 

i 


132 


5 


4 


3 


11 


4 


5 


10 


6 


8 


14 


10 


r 


141 


5 


3 


4 


11 


4 


5 


7 


4 


3 


8 


4 


i 

4 


191 


6 


4 


4 


6 


6 


5 


7 


6 


9 


8 


10 


1- 

| 


195 


4 


9 


7 


8 


7 


6 


5 


5 


10 


12 


5 


196 


6 


7 


3 


4 


5 


6 


5 


14 


4 


3 


9 


i 

> 


202 


2 


2 


6 


11 


5 


4 


6 


10 


2 


4 


11 


226 


3 


4 


4 


3 


5 


4 


5 


3 


8 


8 


9 


i 

i 


239 


3 


9 


11 


11 


2 


5 


3 


6 


3 


5 


3 


i 

r 


247 


2 


2 


4 


12 


4 


3 


3 


6 


4 


5 


12 


y 

t. 


249 


5 


5 


6 


8 


5 


4 


5 


5 


6 


8 


5 


if 


261 


6 


6 


23 


7 


5 


11 


8 


9 


12 


17 


10 


i 


264 


3 


4 


4 


4 


5 


5 


5 


6 


21 


15 


24 




268 


13 


6 


16 


12 


13 


20 


66 


36 


23 


45 


59 




269 


6 


4 


7 


11 


6 


6 


8 


5 


15 


14 


17 


| 


271 


1 


5 


6 


8 


5 


5 


7 


9 


13 


10 


13 




272 


13 


8 


7 


4 


11 


13 


14 


20 


25 


21 


15 


| 


273 


2 


5 


9 


5 


5 


6 


8 


5 


17 


21 


15 


\ 


274 


3 


3 


7 


11 


3 


5 


13 


31 


13 


15 


13 




277 


4 


4 


8 


6 


11 


11 


11 


10 


9 


10 


14 


[• 


278 


2 


5 


7 


4 


5 


5 


8 


5 


17 


16 


10 


[V 


282 


5 


4 


5 


3 


7 


5 


5 


5 


8 


4 


4 


V 


303 


5 


5 


8 


7 


12 


7 


26 


37 


15 


19 


16 




304 


4 


4 


5 


6 


3 


5 


5 


3 


6 


2 


1 




307 


5 


5 


5 


4 


4 


6 


5 


6 


6 


5 


3 




310 


5 


5 


13 


5 


20 


12 


19 


7 


15 


13 


8 


V 


312 


3 


4 


5 


9 


6 


2 


5 


13 


13 


13 


8 


r 


322 


3 


6 


22 


11 


14 


8 


64 


26 


24 


22 


38 


> 

r 

Sfr 


331 


5 


8 


12 


6 


5 


6 


9 


24 


9 


7 


10 


332 


5 


4 


10 


43 


18 


22 


21 


67 


19 


14 


19 


!•. 


334 


4 


5 


13 


21 


7 


8 


12 


36 


15 


10 


20 




336 


5 


4 


2 


11 


5 


5 


7 


6 


4 


8 


6 


|n'. 


338 


2 


5 


9 


3 


3 


5 


5 


4 


8 


6 


5 


£> 

p-. 


Median 


4.0 


4.5 


6.2 


7.2 


5.1 


5.3 


6.5 


6.4 


8.8 


9.8 


10.2 



% 

ERIC 



IS 
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Table 2 

READING SCORES FOR TITLE I SCHOOLS 
Stanford Reading Test National Percentiles 
(For Same Groups) 



I 




School 

Code 

103 

106 

107 

116 

120 

126 

127 

128 
132 
141 
191 

195 

196 
202 
226 
239 
247 
249 
261 
264 
268 
269 

271 

272 

273 

274 

277 

278 
282 

303 

304 
307 
310 
312 
322 

331 

332 
334 
336 
338 



Grade 

One Two Three 
1966 1967 1968 



2 


3 


7 


2 


7 


4 


5 


5 


15 


2 


4 


5 


3 


4 


14 


7 


7 


15 


3 


5 


6 


2 


2 


9 


5 


5 


14 


5 


5 


8 


6 


5 


8 


4 


6 


12 


6 


6 


3 


2 


4 


4 


3 


4 


8 


3 


5 


5 


2 


3 


5 


5 


4 


8 


6 


11 


17 


3 


5 


15 


13 


20 


45 


6 


6 


14 


1 


5 


10 


13 


13 


21 


2 


6 


21 


3 


5 


15 


4 


11 


10 


2 


5 


16 


5 


5 


4 


5 


7 


19 


4 


5 


2 


5 


6 


5 


5 


12 


13 


3 


2 


13 


3 


8 


22 


5 


6 


7 


5 


22 


14 


4 


8 


10 


5 


5 


8 


2 


5 


6 



4.0 



5.3 



Grade Grade Grade 



One 


Two 


Three 


One 


Two 


One 


1967 1968 1969 


1968 


1969 


1969 


3 


5 


15 


5 


8 


5 


2 


5 


12 


3 


9 


10 


7 


6 


11 


6 


6 


11 


2 


5 


8 


5 


5 


4 


4 


5 


11 


3 


8 


7 


5 


12 


9 


6 


8 


4 


5 


5 


10 


9 


5 


6 


1 


3 


4 


6 


5 


8 


4 


10 


10 


3 


6 


11 


3 


7 


4 


4 


4 


11 


4 


7 


10 


4 


6 


6 


9 


5 


5 


7 


5 


8 


7 


5 


9 


3 


14 


4 


2 


6 


11 


6 


10 


11 


4 


5 


9 


4 


3 


3 


9 


3 


3 


11 


6 


11 


2 


3 


12 


4 


6 


12 


5 


5 


5 


6 


5 


8 


6 


8 


10 


23 


9 


7 


4 


5 


24 


4 


6 


4 


6 


66 


59 


16 


36 


12 


4 


8 


17 


7 


5 


11 


5 


7 


13 


6 


9 


8 


8 


14 


15 


7 


20 


4 


5 


8 


15 


9 


5 


5 


3 


13 


13 


7 


31 


11 


4 


11 


14 


8 


10 


6 


5 


8 


10 


7 


5 


4 


4 


5 


4 


5 


5 


3 


5 


26 


16 


8 


37 


7 


4 


5 


1 


5 


3 


6 


5 


5 


3 


5 


6 


4 


5 


19 


8 


13 


7 


5 


4 


5 


8 


5 


13 


9 


6 


64 


38 


22 


26 


11 


8 


9 


10 


12 


24 


6 


4 


21 


19 


10 


67 


43 


5 


12 


20 


13 


36 


21 


4 


7 


6 


2 


6 


11 


5 


5 


5 


9 


4 


3 


4.5 


6.5 


10.2 


6.2 


6.4 


7.2 



16 



Median 



9.8 
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focus their attention on those areas, rather than disperse it over a 
large spectrum. They will want to find out why something is working 
(or not working) in an unusual fashion. If it's extraordinarily good, 
why? Is it because of a teacher with a new, exciting approach who is 
providing in-service training for others in his school, or an extremely 
able administrator whose support encourages teachers to do an outstand- 
ing job? If the results were lower than anticipated, the principal or 
group administrator would still need to determine the cause. 

DATA CATEGORIES 

Achievement data have been briefly discussed in illustrating the 
use of evaluation as a management tool. Alone, achievement data tell 
the decisionmaker almost nothing. Data that describe the total school 
environment are needed so that the decisionmaker may know what programs 
work for whom, and what resource mix is required to implement them. He 
needs data on which to base decisions about program adoption or curtail- 
ment. These data requirements have been divided into six categories 
for discussion: school descriptors, students, teachers, other staff, 

administrators, and classroom descriptors. 

SCHOOL DESCRIPTORS 

Because a strong relationship has been established between socio- 
economic status and achievement, it is necessary to describe the popu- 
lation served by a school. Socio-economic indices should be derived 
to do this. They need not be complicated, but should reflect the most 
prevalent salary range, percent of students receiving Aid for Families 
with Dependent Children, and ethnic composition of the school. Addi- 
tionally, some indication of the school’s mobility rate should be made. 
If schools are judged by the educational attainment of their students, 
and a high percentage of them are not in the school for long enough to 
benefit from its programs, the school cannot be held responsible. 

Finally, if it is a Title I school, data specific to Title I eval- 
uation should be collected so that comparability requirements will be 
met. 




Student Data 



In the final analysis, it is the educational attainment of the stu- 
dents that determines the school's success in meeting its goals. The 
word attainment rather than achievement is consciously used here because 
of its broader connotation. The goals of a school are not only to en- 
courage academic achievement, but also to influence many facets of the 
affective development of the individual. Unfortunately, the state of 
the art of educational measurement is such that the only area that can 
be evaluated with any degree of confidence is achievement. Even here, 
by focussing on gain, the measures are open to serious question. But 
state guidelines (which are constrained by Federal guidelines) require 
fall and spring testing of Title I participants, so the data must be 
collected. These are specified in the requirements for Title I evalua- 
tion. In general, district-wide or state-wide (at least in California) 
testing programs specify the administration of a standardized test only 
once a year, and thus avoid the pitfalls of measuring achievement gain. 
The current testing program for Los Angeles City Schools is given in 
Table 3. 

Age, grade, and sex for each student should be included. Number 
and kinds of counseling contacts a student has had, referrals for dis- 
ciplinary infractions, and attendance data can be used as surrogates 
for attitude, although this does not mean that the absence of recorded 
behavioral problems or of disciplinary referrals indicates a good atti- 
tude toward school or learning. Lacking reliable measuring instruments,, 
however, we can only suggest that at least those students with demon- 
strably poor attitudes be identified. Health problems that may be suf- 
ficiently disabling to interfere with learning should be noted. 

The inclusion of student police contacts in the information system 
involves a policy decision by the school district. If the decision is 
to include these data, the question of access to the files may need re- 
view. Police contact is simply mentioned for consideration as a possi- 
ble indicator of student attitude. 

A student's teachers need to be identified. In the elementary 
school with self-contained classrooms this is not much of a problem. 

In those elementary schools with team teaching in ungraded classrooms. 
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Table 3 

TESTING PROGRAM FOR LOS ANGELES CITY SCHOOLS 





State 


District 




Grade 


Tes ts 


Tests 


When Given 


K 

1 


Reading 


_ 


Spring 


2 


Reading 


— 


Spring 


3 


Reading 


— 


Spring 


4 

5 


— — 


Reading-Intelligence 


Spring 


6 


Reading-Intelligence 


— 


November 


7 


— 


Reading-Intelligence 


Spring 


8 


— — — 


— 


— 


9 


Reading- Ari thine t ic a 


— 


October 


10 


Reading-Arithmetic 3 


— 


Spring 


11 


Reading-Arithmetic 3 


— 


Spring 


12 


Intelligence 


— 


November 


12 


Language 


— 


November 


12 


Spelling 


— 


November 


12 


Mathematics 


— 


November 


12 


Reading 


— 


November 



Not mandated, but to satisfy state requirement. 



multiple entries will be required similar to those needed for junior 
high and high schools in which students have several teachers. But if 
the classroom is to be the basic unit for assessment of educational at- 
tainment, the decisionmaker must know what teaching personnel have been 
in contact with what students. In the same vein, the programs to which 
a student is assigned should be noted. This will be dealt with in more 
detail in the discussion of classroom descriptors. 

Teacher Data 

It is particularly difficult to specify what information about 
teachers is sufficiently meaningful to be routinely and systematically 
collected. The voluminous literature on teacher characteristics has 
failed to yield guidelines about those attributes of a teacher that are 
most likely to promote academic attainment for -his students. It is 



beyond the scope of this report to summarize these findings. They are 

* 

treated in another report in this series. 

There are, however, two reasons for collecting data related to 
classroom teachers even though they cannot be related to success in the 
classroom at present. First, if the data are readily available and if 
a breakthrough is made in specifying salient background characteristics, 
it will avoid a crash effort in the future to collect the data and to 
relate them meaningfully to student's success. Secondly, background 
data on teachers can be used to good advantage in staff planning. 

The need for data on age, sex, and credentials is obvious. Ethnic 
background, however, may turn out to be an important variable. All 
other things being equal, is there any evidence that students taught by 
a teacher of the same race do better (or worse) than students taught by 
a teacher of a different race? The value of experience as a predictor 
of teaching success has not yet been demonstrated (nor has the value of 
education or salary). It may be, however, that certain aspects of ex- 
perience are pertinent. It is suggested, therefore, that experience 
be subdivided into experience in another district, experience in other 
schools within the district, and experience in a teacher's current 
school. In addition, the subjects taught in each of those situations 
should be noted. The implications of this breakdown for evaluation are 
as follows: If there are several teachers all with the same length of 

service, and they achieve quite different results in the classroom, an 
anomalous situation has to be resolved in trying to relate success to 
experience. On the other hand, given the same teachers, but knowing 
that some are new to a school, others are teaching a subject or grade 
level or kind of student for the first time, while still others are 
teaching a familiar curi iculum for the n year, it might be possible 
to identify the element in experience that is important to teaching 
success . 

Two other kinds of data that should be available have current ap- 
plicability to program planning and might ultimately prove useful for 
program evaluation, namely, information about teachers' interests and 
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special abilities. There is no reason to suspect that teachers differ 
from other professionals in preferring an assignment in their field of 
interest. A teacher with demonstrated capability in two areas might 
well be expected to perform better in whichever area he has more inter- 
est. In addition, many teachers have special abilities that could be 
capitalized on in staffing programs, for example, bi-lingua 1 ism, work- 
shop experience, training in new teaching techniques, and the like. If 
these kinds of data were routinely available to the program planner, 
his staffing problems might be diminished. 

The information system facilitates the routine collection of data 
now that may have a pay-off in the future and the routine collection of 
data that may be helpful in program planning and implementation. 

Other School Staff Data 

These data describe the rest of the instructional staff, including 
special teachers and paraprofessionals , counselors, and where appropri- 
ate to program evaluation, data about school nurse and/or physician. 

The rest of the instructional staff includes special teachers not as- 
signed full-time to the school, such as an English resource teacher or 
a mathematics specialist. The same information should be available 
about them as about other teachers. In addition, data .regarding the 
teachers or students they work with and how long they work with them 
should be recorded, as one basis for assessing the impact of such added 
resources on a program. Appropriate background data about paraprofes- 
sionals should be available also. 

Data on full-time counselors, should include at least a minimum of 
information about their case load as well as the same background data 
required for teachers. Number of students seen per week is a rough in- 
dex of the efficiency to be expected. Obviously a counselor cannot do 
more than a superficial job if he sees too many students. If a vice- 
principal also acts as counselor, the proportion of his time spent in 
counseling should be noted. 

Perhaps in the long run, it will be necessary to get much more de- 
tailed information than this for a complete evaluation of programs. 

For Example, data about the proportion of time a counselor spends in 

V, I t > 

,, „ ‘‘l . 

21 
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programming students; in disciplining students; i.n vocational, educa- 
tional, and personal counseling; and in staff and parent conferences 
will be necessary. These are mentioned now, not because it is currently 
feasible to accomplish this kind of task, but because it is seen as a 
very likely requirement in the future. 

Where appropriate, data about the school nurse and/or physician 
will be required. In programs that rely on improving the physical 
health of a student as a means to improving overall performance, the 
special services a child receives must be evaluated. Perhaps background 
data about the nurse and/or physician are secondary to the requirement 
that records of the frequency and type of service they perform can be 
readily related to the records of the students for whom they were per- 
formed. 

Administrator Data 

For the principal and vice-principal the same kinds of data as for 
teachers are required. If effective schools are to be identified so 
that they can be used as models for less effective schools, the effect 
of the administrator cannot be overlooked. Here again the literature 
abounds in references to outstanding administrators but often fails to 
define their important characteristics. As with teachers, the current 
state of the art may not be sufficient to describe the attributes of 
the successful administrator. For evaluative purposes, the use of these 
data may be beyond the state of the art, but for effective staff plan- 
ning, these data are required today. 

Classroom Descriptors 

Without a relatively detailed description of the particular envi- 
ronment in which learning takes place, the educational planner does 
not have sufficient information about programs to make the necessary 
decisions relating to their adoption, expansion, or curtailment. The 
data detailed under the category of classroom descriptors are needed, 
both to assess current programs and to provide a baseline against which 
to measure change. 

22 
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The instructional staff for each classroom should be known — that 
is, the classroom teacher, any special or resource teachers, and the 
presence or absence of paraprofessionals. If their identification des- 
ignators are known, their other characteristics can be described, as 
outlines in the data requirements for teachers and other instructional 
staff. 

In the self-contained classroom, one set of descriptors will suf- 
fice for all the children assigned to it. For ar.y other situation, 
such as team teaching at the elementary level or departmentalized in- 
struction at the secondary level, each classroom to which a student is 
assigned will need to be identified and described. 

Class size, by room, should be noted. The kind of grouping within 
the classroom is an important indicator. Whether the group is homoge- 
neous or heterogeneous, the range of ability within the classroom, and 
in the case of homogeneous groups, whether it is an honors, average, 
or slow class should be included. 

A description of the program is necessary. The first distinction 
to be made is whether this is a "standard" district program or a spe- 
cial situation. "Standard" means the generally accepted curriculum for 
a grade, taught by a regularly credentialed teacher. "Special" means 
any officially recognized deviation from that description — such as an 
experimental program. The nature of the difference is important — for 
example, a staffing pattern, a new teaching technique, or a special 
grouping of children. 

In order for the educational planner to know what other resources 
are being used to achieve educational outcomes, the instructional equip- 
ment and supplies used in a program need to be identified. This in- 
cludes a description of the text books, the audio-visual equipment and 
other instructional aids, and the required teaching supplies. These 
data are also currently required in order for districts to address the 
Title I comparability requirement. 
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IV. AN EXAMPLE OF THE USE OF EVALUATION DATA 



Section III discussed the use of evaluation in an accountability 
system as a management tool and provided examples of its role in as- 
sessing educational achievement and identifying exceptions to expected 
performance. The focal point of this section is evaluation's usefulness 
in simultaneously providing the district with information about the 
progress of its Title I programs and in meeting state and national re- 
porting requirements. 

The data specified in the discussion of what the decisionmaker needs 
to know in order to consider alternative program choices are all germane 
to making decisions about Title I programs. If those data have been 
collected and analyzed, they meet the state requirements for Title I re- 
porting. In fact, these data go beyond the requirements in terms of 
detail about students and programs and provide a better basis for deci- 
sions about future operation. 

Additional data are required, however, to meet the new comparabil- 
ity requirements. For each Title I school, it is now necessary, because 
of a 1970 amendment to Title I of the Elementary and Secondary Education 
Act, to show that services and funds comparable to those in non— Title I 
schools are being provided before the addition of Title I funds. In 
other words, this is an attempt to ensure that Title I funds are used 
in addition to, and not as a substitute for, state and local funds. In 
order to continue receiving Title I funds, local educational agencies 
will need to demonstrate that their Title I schools have equal or lower 
personnel ratios and equal or higher expenditures than the average for 
non-Title I schools serving the same grade level. The Comparability 
Status Report and the Worksheet developed for California (Figs. 1 and 
2) show the data required by each school district in order to demon- 
strate comparability-. 

In addition, a special evaluation of Title I projects is required. 
Figure 3 shows the data requirements for Title I evaluation and summa- 
rizes the data necessary for completing the comparability report. 

A flexible information system can easily adapt to special report- 
ing requirements such as this. More important, as better measures of 



+■> 

u 



c 

fO 

CL 

00 



&- (U 

4-> TJ 

CO ITJ 

•r- S- 

Q CD 



C 

o 

•r* 

•P 
ITJ 
O 
=3 

-a 

LU -r- 

vS 

o o 



c 

o 



3 


< 3 * 


•P -o 




C LU 


00 


0 


Lf) 




o> 


S- O 


ITJ 


<TJ -P 


•r* 


a. itj 


C 


0 to 


u 


a c 


o 


0 


4 - 


0 Q.f— 


•r- 



■p . 

ITJ 5 fO fO 

puSu 

00 

4 - 1 — * 

ITJ o o o 
•r- -P *P 
C C -r- C 

o o aa) 

0 ‘r (O P 

4- t/> o <3 

•r- *r- i- 
r— > •— U 
<TJ *r- CM ITJ 
U Q S(/) 



H* 

on 

o 

CL. 



>• 

H 



00 

LU 

on 

O 

H* 

HH 

Q 



a. 

X 



00 _ 

ZD 

h- -J 

5 l “ l 

H a. 

00 30 
a. hs 
i 

QC CT> 
LU VO 
Q, CJ> 



00 



C 

CL. 

s 

o 

o 



Q 

00 QC 
O O 



§ 



j— S 

a: oo 
2 u 

t-H 

Q. 

=3 

Q. 



-18- 



lo 


Per Pupil Ex- 
penditures for 
Other Instruc- 
tional Costs 


* £ ^M-Liqejeduioo 


Yes 

No 


Yes 

No 


aouaja^LQ % 














Per Pupil Ex- 
penditures for 
Sal ari es 


* l £m.iqejediuoo 


Yes 

No 


Yes 

No 


aouaua^ia % 












CO 


Pupil /Non- 
Certi f i cated 
Instructional 
Staff Ratio 


* £ ^iL.iqBjeduioo 


Yes 

No 


Yes 

No 


aouaua^ia % 












CVl 


Pupil /Other 
Certificated 
Personnel 
Rati o 


* £ 'fcUUqeueduioo 


Yes 

No 


Yes 

No 


aouaua^iQ % 












r— 




* £ Uqeueduioo 


Yes 

No 


Yes 

No 


aouaua^iQ % 






■ Pupil/ 
Teache 
Ratio 












. District 
; Average for 
1 Non-Title I 
Schools 




(name) 

Title I School 




(name) 

Title I School 



p 

u 

(U 

•o 

0 
s- 

Q. 

1 

c 

o 

c 

s- 

o 

4 - 

co 

0 

cn 

ITJ 

5- 
0 
> 
ITJ 

0 

JC 

•P 

O 

•P 



ITJ 

3 

or 

CD 

*o 

2 

CD 

“O 

•r* 

l/) 

c 

O 

O 

0 

jQ 



ITJ 

JC 

CO 



o 

o 

JC 

CJ 

CO 



0 -p 
f— c 

P 0 
;r- CJ 
H- S- 
0 

jc a. 
cj 

ITJ 0 
0 > 
•r* 
S- 

£c 

CO JC 

2S 

3* 

•i— QJ 
"O S- 
C ITJ 
0 

Q. >> 

x or 

0 JC 

•p 

•o 

c 4- 

ITJ •»— 

CO CO 
O f— 

* 1 — o 
•P o 

ITJ JC 
C£ <J 
* CO 

ITJ 

0 

s- 

ITJ 



I 






-California State Department of Education comparability status report 



CRITERIA FOR DEMONSTRATING COMPARABILITY 



19- 



*o 

OJ 



o 

o 

-C 

o 

in 

< 4 - 

o 



<o 

Q. 

00 

01 

*o 

u 

C3 



U 

*U 

•M 

Q 



O 

O 



O 

00 



(l ’ l°D * 21 ’ 103 ) 
s^sog leuo^Dru^sui 
J3MT0 -»oi asuadx 3 Ltdnj u a d 
















S 3 S 03 iBuojqoruqs 
-ui 43^0 403 papuadx] qunouiv 
















(l >10D * oi * 103) 
A3)A36U01 SS3T S3)4e)B$ IBU0)3 

-31143 sui 403 3 su 3 dx 3 i)dn<| 43 j 
















, c (6 ' l°D "8 ' 103 ) 

*3)A36U03 SS3T S3)4B)BS )BU0)3 
-an4?sui 403 P3pu3dx3 ^unouiv 
















A}lA36ucn 
J 03 )Ci3ios pspusdxg ^unouiv 
















(^iA36uoi 
6u)pn)3U)} S3 )jb).b$ ibuo )3 
-3n4?sui J 03 P3pu3dx3 ^unouiv 
















(fr • l°3 * l > l°3) 
11B3S lBU0H3n4?SUI P3WHJ3D 
-UON 3 U 03 si)dn<j 30 0)3B)) 
















(E 'l°D * l > 10 D) 
31B3S lBU0H3n4?SUI P3HH433 
431)30 313 03 si)dnj 30 0 ) 3 B )1 
















(2 ’ l°D * l > 103 ) 
$431)3631 UI004SSB13 p3)3 
-)3J83 313 03 sudnj 30 0 ) 36 )) 
















11B3S lBU0)33n43Sui psy 
-)3433-uofj 313 43qunt) 36 b43A\/ 
















33B3S lBU0)33n43SUi P3)3)3J33 
431)30 313 43 qumf) 36 b 43 a\/ 
















S43l)3B31 IU004SSB13 
P3 13)3433 313 43qum)) 36 b43A\/ 
















WOV U) si)dnd 30 43qumf) 
















Title I Project 
Schools (List 
Individually) 














Average for Non- 
Title I Project 
Schools 



u 

t 

3 

CT 

2 



-Q 

2 

<o 

Q. 

E 

8 



O. 

E 

o 

o 

<«- 

o 



0* 

*o 



0* 

U1 

<0 

-Q 



<o 

.c 

C/1 

2 

00 



.c 

u 

•r— 

.C 

2 



o 

cx 



u 

01 



u 

u 



5 

! 

3 



•*4 

+4 

a 



C«5 

44 

C 

1 






5 

<3 

•«» 

I 

I 

I 

co 



* 




26 



- 20 - 



Evaluation 



Component 

School 

Grades 

participants 

Organizational system (Personnel) 

Time allocation 

Days of instruction 
Time per day per student 
Hours of instruction 
Pretest/Posttest data 
Dates 

Name of test (form, level) 

Median grade placement 

Mean raw scores 

Number of students taking both 

Comparabi 1 i ty 

School 

Number of certified teachers 
Number of other certified instructional personnel 
Number of noncertified instructional personnel 
Instructional salaries [length of service increments] 
Other instructional costs 
textbooks 
library books 
A.V. materials 

a.d.a. 



Fig. 3 — Siamary of data requirements for Title I re-porting 
and for completing comparability status report 



stated objectives are developed they can be incoporated into the system 
with minimum disruption. If the collection of program data is a routine 
operation, baseline data is always available against which to compare 
new programs. For example, when an innovative program is tried, it is 
not necessary to institute an additional series of tests; the same data 
are collected for all ongoing district programs. The decisionmaker who 
has detailed knowledge of the kind outlined in this report is in a bet- 
ter position to implement programs that seem to be working for defined 
populations and to curtail those that do not hold much promise. 
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